Occlusion areas are parts of a scene which are visible in one frame and become invisible in a neighboring frame of multiple consecutive images due to blockage by foreground object(s). Each of the consecutive images can be a picture captured by imaging system or an interpolated image based on captured pictures. For consecutive images captured by a camera, the presence of the occlusion areas is caused by the changing of relative position between the objects and the camera. When parts of the image become invisible, at least a portion of a background object is covered by a foreground object located closer to the camera due to the projection. Vice versa, the foreground object is located farther away from the camera in the scene and parts of the background become uncovered. The parts of the background object which become covered or uncovered are referred to as occlusion areas. When the areas become covered or uncovered, the uniquely matched areas in neighboring images cannot be found by motion estimation. Therefore, special care must be taken in addressing the occlusion problem, otherwise artifacts may occur in temporal interpolation.
In a conventional method, occlusion detection is based on pixel differences associated with two motion vectors between two consecutive images (U.S. Pat. No. 7,995,793). One of the two motion vectors may be zero, which corresponds to a background that is stationary relative to the camera. FIG. 1 illustrates an example of occlusion detection based on two motion vectors between two consecutive frames, Frame (t) and Frame (t+1) captured by an imaging system. Frame (t+δ) is to be temporally interpolated based on Frame (t) and Frame (t+1). Two motion vectors (i.e., MV1 and MV2) are determined to describe the motion between the two frames, where MV1=0 corresponds to zero motion for the background areas.
Frame (t+δ) is formed by projecting Frame (t) to Frame (t+1) according to one of the two motion vectors. For example, area 111 in Frame (t+δ) is formed by projecting area 101 in Frame (t) to area 121 in Frame (t+1) according to motion vector MV1 since good match between area 101 and area 121 can be found using motion vector MV1. Similarly, area 112 in Frame (t+δ) is formed by projecting area 102 in Frame (t) to area 122 in Frame (t+1) according to motion vector MV2 since good match can be found between area 102 and 122 using motion vector MV2. Nevertheless, for area 113 in Frame (t+δ), none of the motion vectors can result in good match between corresponding areas in Frame (t) and Frame (t+1). In other words, the pixel differences are large for corresponding areas in Frame (t) and Frame (t+1) associated with area 113 in Frame (t+δ) regardless which of the two motion vectors is used.
While the above method works successfully to a certain degree, it is desirable to develop an occlusion detection method to improve the accuracy of detection and/or reduce associated computational complexity.